Wireless transmission of real-time media

ABSTRACT

A method, wireless communication device, and computer readable medium, are disclosed, for wireless transmission of real-time media from a source to a sink over a wireless transmission channel. The wireless device initiates a peer-to-peer communication session between the sink and the source, and determines based on a time-allocation for the wireless transmission, an available bandwidth for the wireless transmission. The wireless device encodes the real-time media for the wireless transmission such that a time-required for transmission of the first frame-type is less than or equal to a period-of-availability for transmission defined in the time-allocation for the wireless transmission.

TECHNICAL FIELD

The present disclosure relates generally to techniques in wirelessdevices which are configured for wireless transmission of real-timemedia, for example, by using a wireless transmission channel configuredfor Wi-Fi peer-to-peer (P2P) communication.

BACKGROUND

A wireless communication device, such as a portable battery-poweredwireless communication device, may be configured to communicate viaaccess points (APs) of wireless local area networks (WLANs) inaccordance with IEEE 802.11 standards or the like. Such a device mayadditionally communicate using peer-to-peer communication techniques,for example, over a wireless transmission channel configured inaccordance with the “Wi-Fi Direct” technical specification (also knownas Wi-Fi Peer-To-Peer (“Wi-Fi P2P”) technical specification). Such adevice may be certified as a Wi-Fi Direct device.

There is a need for efficiently facilitating real-time mediatransmission over the wireless transmission channel to enable wirelesscommunication devices to transmit and/or receive real-time media to asecond communication device such as a television, for display. Forexample, a portable wireless communication device may have a smallersized display screen than the second communication device.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made, by way of example, to the accompanyingdrawings which show example embodiments of the present application, andin which:

FIG. 1A illustrates an example environment within which the techniquesof the present disclosure can be practiced;

FIG. 1B illustrates an additional example environment within which thetechniques of the present disclosure can be practiced;

FIG. 2 illustrates in block-diagram form a display suitable fordisplaying real-time streaming media in accordance with exampleembodiments of the present disclosure;

FIG. 3 illustrates in block-diagram form a wireless device suitable fortransmitting real-time streaming media to the display for FIG. 2 inaccordance with example embodiments of the present disclosure;

FIG. 4 illustrates a flow-diagram of communication between the wirelessdevice of FIG. 3 and the display of FIG. 2 to establish a peer-to-peersession to allow for wireless transmission of real-time media betweenthe two devices;

FIG. 5 illustrates example encoding schemes for encoding the real-timemedia for wireless transmission in accordance with example embodimentsof the present disclosure;

FIG. 6A illustrates in block-diagram form example components of thewireless device of FIG. 3 and the display of FIG. 2 for transmission ofthe real-time media in accordance with example embodiments of thepresent disclosure;

FIG. 6B illustrates an example packet structure for encapsulating mediafor wireless transmission in accordance with example embodiments of thepresent disclosure;

FIGS. 7A and 7B illustrate example time-allocation schedules;

FIG. 8 illustrates an example flow-chart of a method for wirelesstransmission of real-time media in accordance with example embodimentsof the present disclosure;

with example embodiments of the present disclosure; and

FIG. 9A-9D illustrate example time-allocation schedules and exampletiming diagrams for real-time media transmission.

Similar reference numerals may have been used in different figures todenote similar components.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Real-time media transmission over a wireless transmission channelenables many applications. For example, an Internet-connected portablewireless communication device may stream a video from a video sharingInternet site and wirelessly transmit the video to a television having alarger display screen. In another example, a presentation, video orother media stored in the memory of a portable wireless communicationdevice may be wirelessly transmitted to a projector for presentation toan audience. However, as such applications require real-time or nearreal-time processing and transmission of the media, more efficientencoding of the media is necessary to allow for reduced latency oftransmission, and to occupy the wireless channel more efficiently, thusreducing the bandwidth required for transmission.

The present disclosure teaches a wireless communication device(hereinafter “wireless device” for convenience) having a non-transitorycomputer-readable medium storing instructions for implementing a methodfor encoding real-time media for wireless transmission from a source toa sink over a wireless transmission channel. A processor of the wirelessdevice may implement the method to transmit real-time media bydetermining, based on a time-allocation for the wireless transmission,an available bandwidth for the wireless transmission; and encoding thereal-time media for the wireless transmission such that a time-requiredfor transmission of the first frame-type is less than or equal to aperiod-of-availability for transmission defined in the time-allocationfor the wireless transmission.

In another aspect of the present disclosure, the start of transmissionof the frame of the first frame-type is aligned with the start of theperiod-of-availability.

In another aspect of the present disclosure, the time-allocation isreceived from a group-owner, wherein the time-allocation is defined by aNotice-of-Absence schedule.

In another aspect of the present disclosure, the real-time media isencoded for the wireless transmission based on the available bandwidthor a throughput constraint.

In another aspect of the present disclosure, the processor determinesbased on a hardware limitation associated with the source or the sink,the throughput constraint for the wireless transmission; and when theavailable bandwidth is greater than or equal to the throughputconstraint, encodes the real-time media for the wireless transmissionbased on the throughput constraint.

In another aspect of the present disclosure, the wireless device is agroup-owner and sets the time-allocation using a Notice-of-Absenceschedule.

In another aspect of the present disclosure, the processor encodes thereal-time media for the wireless transmission based on a throughputconstraint; determines, based on the available bandwidth, thetime-required for transmission of a frame of the first frame-type; andsets the time-allocation for wireless transmission, such that theperiod-of-availability for transmission is greater than or equal to thetime-required for transmission of the first frame-type.

In another aspect of the present disclosure, the processor determinesthat a second frame-type is queued for transmission; and sets a newtime-allocation for wireless transmission, such that theperiod-of-availability for transmission is greater than or equal to thetime-required for transmission of the second frame-type.

In another aspect of the present disclosure, the first frame-type is anIntra-Frame, and the second frame-type is a Predicted-Frame.

In another aspect of the present disclosure, the processor initiates apeer-to-peer (P2P) communication session between the sink and thesource.

Example Network Configuration

To illustrate one environment within which the techniques of the presentdisclosure may be practiced, FIG. 1A illustrates a wireless device 130which may communicate with wireless communication devices 120 a, 120 band 120 c (120 collectively) and an access point (AP) 125. Certainadaptations and modifications of the described embodiments can be made.Therefore, the above discussed embodiments are considered to beillustrative and not restrictive. The wireless device 130 maycommunicate with one or more wireless communication networks. Forexample, wireless device 130 may communicate with a wireless local areanetwork (WLAN) 110 operating in accordance with IEEE 802.11 standards,or other WLAN standards, and the Internet 115 via AP 125.

The wireless device 130 additionally or alternatively communicates usingwireless peer-to-peer communication techniques, for example, inaccordance with the Wi-Fi Direct technical specification (also known asWi-Fi Peer-To-Peer (“Wi-Fi P2P”) technical specification) and/or becertified as a Wi-Fi Direct device. The wireless device 130 mayestablish a Wi-Fi P2P wireless connection with a display 120 a (ormonitor) which includes a wireless transceiver. Such a Wi-Fi P2Pwireless network connection may be suitable for applications such as,for example, a streaming media application, or a display or presentationapplication. The wireless device 130 may additionally or alternativelyestablish a Wi-Fi P2P wireless network connection with a printer 120 bwhich includes a wireless transceiver. Such a Wi-Fi P2P wireless networkconnection may be suitable for applications such as, for example, aprint application, or a facsimile application. Even further, thewireless device 130 may additionally or alternatively establish a Wi-FiP2P wireless network connection with a speaker 120 c which includes awireless transceiver. When the wireless device 130 is connected as such,using one or more Wi-Fi P2P wireless network connections, data may becommunicated “directly” between the wireless device 130 and the otherdevices 120 (i.e. without the data traversing any fixed wireless networkinfrastructure).

Wi-Fi P2P wireless networks may include a P2P wireless device which isdesignated as a group-owner (GO) to serve some functions of an AP, suchas, broadcasting beacon frames and allocating wireless channelresources. The GO may maintain multiple concurrent network connectionsin an active state, for example, with multiple devices, including thewireless devices 120 and the AP 125. In FIG. 1A, the wireless device 130acts as a group-owner.

The wireless device 130 may be additionally configured to accesscommunication services via a Public Land Wireless Network (PLWN) (notshown), such as a cellular telecommunications network. For communicationwith PLWNs, the wireless device 130 may be configured in accordance withone or more cellular telecommunication standards, such as Global Systemsfor Mobile (GSM), General Packet Radio Service (GPRS), Enhanced Datarates for GSM Evolution (EDGE) or Enhanced GPRS (EGPRS), UniversalMobile Telecommunications System (UMTS), Long-Term Evolution (LTE), orEVolution-Data Only (EV-DO) (for CDMA) technologies, as a few examples.

FIG. 1B illustrates a second environment within which the techniques ofthe present disclosure may be practiced. In FIG. 1B, the display 120 aacts as a group-owner. The display 120 a may maintain a networkconnection in an active state, for example, with multiple devices,including the wireless device 130 and the AP 125, concurrently. Thedisplay 120 a may connect to the Internet 115 via the AP 125.

Example Display Device

Reference is next made to FIG. 2 which shows in block-diagram form anexample of the display 120 a suitable for displaying real-time streamingmedia, such as video and presentations in accordance with exampleembodiments of the present disclosure. The display 120 a may be any oneof a television, a projector, a computer monitor, an adaptor coupled toa display, or other device suited for displaying information on adisplay screen.

The display 120 a includes a rigid case (not shown) housing theelectronic components of the display 120 a. The electronic components ofthe display 120 a are mounted on a printed circuit board (not shown).The display 120 a includes a processor 202 which controls the overalloperation of the display 120 a and a communication interface 204 forcommunicating with other devices via a communication network 150. Thecommunication network 150 may be the network shown in FIG. 1A or 1B, orother suitable communication network.

The processor 202 interacts with other components, such as one or moreinput devices 206 such as a keypad, buttons, or touch sensitive bezel,Random Access Memory (RAM) 208, Read Only Memory (ROM) 210, a displayscreen 212, persistent (non-volatile) memory 220 which may be flasherasable programmable read only memory (EPROM) memory (“flash memory”)or any other suitable form of memory, auxiliary input/output (I/O)subsystems 250, one or more data ports 252 such as a serial data port(e.g., Universal Serial Bus (USB) data port and High-DefinitionMultimedia Interface (HDMI) data port, a speaker 256, and other devicesubsystems generally designated as 264. The components of the display120 a are coupled via a communications bus (not shown) which provides acommunication path between the various components.

The display screen 212 may be provided as part of a touchscreen whichprovides an input device 206. The display screen 212 which together witha touch-sensitive overlay (not shown) operably coupled to an electroniccontroller (not shown) comprise the touchscreen. User-interaction with aGUI (graphical user interface) is performed through the input devices206. Information, such as text, characters, symbols, images, icons, andother items are rendered and displayed on the display screen 212 via theprocessor 202.

The processor 202 operates under stored program control and executessoftware modules 276 stored in memory, for example, in the persistentmemory 220. The persistent memory 220 stores data 286 such as user data.As illustrated in FIG. 2, the software modules 276 comprise operatingsystem software 278 and software applications 280. The softwareapplications 280 include a P2P Streaming application 282. The softwaremodules 276 or parts thereof may be temporarily loaded into volatilememory such as the RAM 208. The RAM 208 is used for storing runtime datavariables and other types of data or information. Although specificfunctions are described for various types of memory, this is merely oneexample, and a different assignment of functions to types of memorycould be used.

The communication interface 204 may include a short-range wirelesscommunication subsystem (not shown) which provides a short-rangewireless communication interface. The short-range wireless communicationinterface may be configured in accordance with one or more cellulartelecommunication standards, including any one of a Bluetooth® standard,an IEEE 802.11 standard, an IEEE 802.15.3a standard (also referred to asUltraWideband (UWB)), a Z-Wave standard, a ZigBee standard or othersuitable short-range wireless communication standard. The communicationinterface 204 may provide an infrared (IR) interface such as an InfraredData Association (IrDA) interface to receive communication from a remotecontrol unit (not shown) for controlling operation of the display 120 a.

The P2P streaming application 282 configures the display 120 a todisplay information received via the communication interface 204 over aP2P wireless network, such as a Wi-Fi P2P wireless network, on thedisplay screen 212. The information may be processed in real-time ornear real-time, and may include video, audio, pictures, text, anycombination of audio, pictures and text, or other media or multimedia.In some embodiments, the P2P streaming application 282 enables thedisplay 120 a to act as an external display device or monitor for aconnected computing device such as the wireless device 130, includingcloning another display device such as the display screen of thewireless device 130, or acting as a primary display device or monitorfor the connected computing device. The P2P streaming application 282may additionally or alternatively receive audio from the wireless device130 as part of the real-time or near real-time information, which may bereproduced using the speaker 256 of the display 120 a or an externalspeaker (not shown) coupled directly or indirectly to the display 120 a.

The P2P streaming application 282 may run in the background,concurrently with another application, such as a TV application (notshown). Accordingly, the P2P streaming application 282 may be triggeredupon detecting a new P2P connection has been established with a devicesupporting P2P streaming, such as the wireless device 130.

Example Communication Device

Reference is next made to FIG. 3 which illustrates a mobile wirelessdevice 130 suitable for communicating with the display 120 a inaccordance with example embodiments of the present disclosure. Examplesof the wireless device 130 include, but are not limited to, a mobilephone, smartphone or superphone, tablet computer, notebook computer(also known as a laptop, netbook or ultrabook computer depending on thedevice capabilities), wireless organizer, personal digital assistant(PDA), electronic gaming device, and digital camera.

The wireless device 130 includes a rigid case (not shown) housing theelectronic components of the wireless device 130. The electroniccomponents of the wireless device 130 are mounted on a printed circuitboard (not shown). The wireless device 130 includes a processor 302which controls the overall operation of the wireless device 130.Communication functions, including data and voice communication, areperformed through a communication interface 304. The communicationinterface 304 receives messages from and sends messages via thecommunication network 150. The communication interface 304 typicallyincludes a WWAN interface for communication over cellular networks and aWLAN interface for communication over Wi-Fi networks.

The processor 302 interacts with other components including one or moreinput devices 306 such as a keyboard and/or touchscreen, RAM 308, ROM310, a display screen 312, persistent (non-volatile) memory 320 whichmay be flash memory or any other suitable form of memory, auxiliary I/Osubsystems 350, one or more data port 352 such as serial data port(e.g., USB data port), a camera 354 such as video and/or still camera, aspeaker 356, a microphone 358, a motion sensor 368 which enables toprocessor 302 to determine whether the wireless device 130 is in motionand the nature of any sensed motion at any appropriate time, anorientation sensor 370 which enables the processor 302 to determinewhich direction the wireless device 130 is pointed at any appropriatetime, a global positioning system (GPS) device 372 which enables theprocessor 302 to determine GPS coordinates (i.e., location) of thewireless device 130 at any appropriate time, proximity sensor 374 whichenables the processor 302 to determine the distance between the wirelessdevice 130 and an object at any appropriate time, and other devicesubsystems generally designated as 364. The components of the wirelessdevice 130 are coupled via a communications bus (not shown) whichprovides a communication path between the various components.

The display screen 312 may be provided as part of a touchscreen whichprovides an input device 306. The display screen 312 which together witha touch-sensitive overlay (not shown) operably coupled to an electroniccontroller (not shown) comprise the touchscreen. User-interaction with aGUI is performed through the input devices 306. Information, such astext, characters, symbols, images, icons, and other items are renderedand displayed on the display screen 312 via the processor 302. Theprocessor 302 may interact with the orientation sensor 370 to detectdirection of gravitational forces or gravity-induced reaction forces soas to determine, for example, the orientation of the wireless device 130in order to determine a screen orientation for the GUI.

The input devices 306 may include a keyboard, control buttons (notshown) such as a power toggle (on/off) button, volume buttons, camerabuttons, general purpose or context specific buttons, ‘back’ or ‘home’buttons, phone function buttons, and/or a navigation device. When thedisplay screen 312 is provided as part of a touchscreen, the variousbuttons or controls may be provided by onscreen user interface elementsdisplayed on the display screen 312 instead of, or in addition to,physical interface components. The keyboard may be provided instead of,or in addition to, a touchscreen depending on the embodiment. At leastsome of the control buttons may be multi-purpose buttons rather thanspecial purpose or dedicated buttons.

The wireless device 130 may include a memory card interface 330 forreceiving a removable memory card 332 comprising persistent memory, suchas flash memory. A removable memory card 332 can be inserted in orcoupled to the memory card interface 330 for storing and reading data bythe processor 302 including, but not limited to still images andoptionally video images. Other types of user data may be stored on theremovable memory card 332. Other types of removable digital imagestorage media, such as magnetic hard drives, magnetic tape, or opticaldisks, may be used in addition to, or instead of, the removable memorycard 332.

The processor 302 operates under stored program control and executessoftware modules 375 stored in memory, for example, in the persistentmemory 320. The persistent memory 320 stores data 386 such as user data,user information and information regarding the components and technicalcapabilities of the wireless device 130. As illustrated in FIG. 3, thesoftware modules 376 comprise operating system software 378 and softwareapplications 380. The software applications 380 may include a P2PStreaming application 382. The software modules 376 or parts thereof maybe temporarily loaded into volatile memory such as the RAM 308. The RAM308 is used for storing runtime data variables and other types of dataor information. Although specific functions are described for varioustypes of memory, this is merely one example, and a different assignmentof functions to types of memory could be used.

The communication interface 304 may include a short-range wirelesscommunication subsystem (not shown) which provides a short-rangewireless communication interface. The short-range wireless communicationinterface may be configured in accordance with one or more cellulartelecommunication standards, including any of a Bluetooth® standard, anIEEE 802.11 standard, an IEEE 802.15.3a standard (also referred to asUWB), a Z-Wave standard, a ZigBee standard or other suitable short-rangewireless communication standard.

The P2P streaming application 382 configures the wireless device 130 toinitiate communication with a display, such as display 120 a, over awireless P2P network and to send information to the display over thewireless P2P network via the communication interface 304. Theinformation may be stored in the memory 320, for example as persistentdata 386, or on a removable memory card 332, or may be retrieved fromthe Internet 115 in real-time or near real-time, or may be generated bythe processor 302. The information may be processed in real-time or nearreal-time, and may include video, audio, pictures, text, any combinationof audio, pictures and text, or other media or multimedia. In someembodiments, the P2P streaming application 282 enables the wirelessdevice 130 to “clone” the content displayed on the display screen 312 onthe display 120 a, or to use the display 120 a as a secondary displaydevice. The P2P streaming application 282 may additionally oralternatively transmit audio from the wireless device 130.

The P2P streaming application 282 may run in the background,concurrently with another application, such as an Internetvideo-streaming application (e.g. YouTube®). Accordingly, the P2Pstreaming application 282 may be triggered upon detecting launch of thevideo streaming application.

The wireless device 130 includes a battery 338 as a power source, whichis typically one or more rechargeable batteries that may be charged, forexample, through charging circuitry coupled to a battery interface suchas the serial data port 352. The battery 338 provides electrical powerto at least some of the electrical circuitry in the wireless device 130,and the battery interface 336 provides a mechanical and electricalconnection for the battery 338. The battery interface 336 is coupled toa regulator (not shown) which provides power V+ to the circuitry of thewireless device 130.

A received signal, such as a text message, an e-mail message, or webpage download, is processed by the communication subsystem 304 and inputto the processor 302. The processor 302 processes the received signalfor output to the display screen 312 and/or to the auxiliary I/Osubsystem 350. A subscriber may generate data items, for example e-mailmessages, which may be transmitted over the communication network 150through the communication subsystem 304, for example.

The motion sensor 368 may comprise an accelerometer (such as athree-axis accelerometer) or other suitable motion sensor. Theorientation sensor 382 may comprise an accelerometer (such as athree-axis accelerometer), electronic compass, gyroscope, or acombination thereof. Other suitable orientation sensors could be usedinstead of, or in addition to, the accelerometer, electronic compass andgyroscope. The motion sensor 368 and orientation sensor 382, or partsthereof, may be combined or shared, for example, within an integratedcomponent. The processor 302, or controller (not shown) of a three-axisaccelerometer, can convert acceleration measurements into deviceorientations.

The proximity sensor 374 may comprise a sensor that transmits a field orsignals (such as electromagnetic) to detect the presence of nearbyobjects (i.e. the sensor's target). The maximum distance that theproximity sensor 374 can detect may be predetermined or adjustable. Theprocessor 302 can utilize this information to determine the distancebetween the wireless device 130 and the target object to be captured inan image.

Session Establishment

FIG. 4 illustrates an example flow-diagram of communication between thewireless device 130 and the display 120 a to establish a P2P session forwireless transmission of real-time media between the two devices. Theflow-diagram of FIG. 4 provides only a high-level illustration of stepsand messages that may be communicated to establish a session. Variousother steps may be implemented and various other messages may becommunicated. Additionally, the order of the steps and messages is onlyillustrative and is non-restrictive.

The wireless device 130 and the display device 120 a may be configuredto scan for other P2P available devices at 402. The wireless device 130and the display device 120 a may receive an instruction to scan from auser via input received via a user interface of the wireless device 130or the display device 120 a, or may be programmed to perform a scan whena pre-determined condition is detected. The pre-determined condition maybe the launch of a particular application, such as a video application.The scanning procedure allows the devices 130, 120 a to discover eachother at 404, and negotiate parameters for selection of a wirelesschannel, such as a channel number.

After the devices 130, 120 a have discovered each other, the devices130, 120 a may enter into a group-owner (GO) negotiation phase at 406.The GO negotiation allows for the selection of one of the devices 130,120 a to act as a GO to perform functions similar to that of an AP in atraditional Wi-Fi network. The selection of the GO may be based on manyfactors, including factors related to IT policy, the available services,interference with other wireless devices and the ability to access othernetworks. However, battery-constrained devices often act as the GO. TheGO may select and establish a Notice-of-Absence (NoA) schedule defining“absence” periods during which the GO may enter an inactive state, suchas a low-power state in which wireless communication functions aresuspended. The NoA schedule may be broadcast in a beacon frame by the GOat regular intervals. The NoA schedule defines a time-allocation foreach device to transmit over the wireless channel using four parameters:(1) a time-duration parameter, specifying the length of each absenceperiod; (2) a time-interval parameter, specifying the time betweenconsecutive absence periods; (3) a start-time, specifying the startingtime of the first absence period after the current beacon frame; and (4)a count of the number of absence periods in the current schedule. At theend of each absence period, the GO returns to an active state from theinactive state, for example, when changing from the low-power state to ahigher-power state, such as a normal operating state. The GO may adjustthe NoA schedule at any time.

The basis upon which a particular NoA schedule chosen may be based onfactors such trying to minimize latency while satisfying a certainthroughput/power consumption tradeoff. For example, when the GO is alsocommunicating with an infrastructure access point, the one interface(e.g., P2P) must work around the other interface (e.g., theinfrastructure). The implementations are often proprietary.

After the devices 130, 120 a have discovered each other, the devices130, 120 a may enter into a device capability negotiation phase at 408.The device capability negotiation may include exchanging messagesproviding details of supported compression schemes and standards and/orother device capability information. For example, when the H.264 MPEG-4video compression standard is used, the devices 130, 120 a may exchangeinformation regarding supported profiles and/or levels. Each profiledefines a particular set of features to be supported, and each profileis tailored to a specific class of applications. For example, theConstrained Baseline Profile (CBP) defines a low-cost set of features,suitable for videoconferencing and mobile applications. In anotherexample, the High Profile (HiP) supports high-definition televisionapplications. Example features that are be supported by HiP but not CBPinclude: 10 bit sampling; interlaced coding; and quantization scalingmatrices. Additionally, each level may include definitions of: maximumdecoding speed; maximum frame size; maximum video bit rate for coding;and maximum resolution. Accordingly, to support a particular profile andlevel, a particular set of hardware and software performancerequirements may be needed.

A profile may be selected based on the type of media being transmitted.For example, when transmitting a movie for display on the display 120 a,the HiP may be selected. However, the level may be selected based onhardware limitations associated with either of the devices 130, 120 a.In one embodiment, the level selected may be the level providing themaximum image and/or video quality given the hardware performanceconstraints of the devices. In another embodiment, the level selectedmay be the level providing the maximum battery life for the devices 130,120 a.

In one embodiment, the hardware limitation is a buffer limitation of thedisplay 120 a for storing received data from the wireless device 130 ora decoder limitation of a decoder of display 120 a for decoding receiveddata from the wireless device 130. In another embodiment, the hardwarelimitation is a buffer limitation of a buffer of the wireless device 130for storing data for sending to the display 120 a or an encoderlimitation of an encoder of the wireless device 130 for encoding datafor sending to the display 120 a. In another embodiment, the decoderand/or the buffer limitations may be dependent on the softwareimplementation of the decoder and buffer respectively. For example, if amore efficient algorithm is used by the decoder, the hardwarelimitations associated with the decoder may be reduced.

During the device capability negotiation phase, the devices 130, 120 aidentify a maximum average throughput which the devices 130, 120 a canboth support due to hardware limitations associated with the wirelessdevice 130 and/or the display 120 a. The hardware limitations maydirectly affect the ability of the wireless device 130 to process andoutput real-time (or near real-time) media and for the display 120 a toprocess and display the real-time media without interruptions. Thehardware limitations add latency to the system which may delay thedisplay of a frame or frames of the real-time media on the display 120a. This may be considered to be unacceptable, as each frame has aspecific time at which it must be displayed, to ensure continuity of thereal-time media.

A session may be established between the wireless device 130 and thedisplay 120 a after completing the negotiation at 410. The devices mayin some embodiments use a Real Time Transport Protocol (RTP) overTransmission Control Protocol (TCP) or User Datagram Protocol (UDP) asthe communication protocol for sending and receiving data packets duringthe session. Accordingly, the wireless device 130 may prepare the mediacontent for transmission by encoding the media into data packets usingthe negotiated compression parameters and encapsulate the encoded datapackets into a data frame, as explained with reference to FIGS. 6A and6B.

Overview of Wireless Media Transmission

FIG. 6A illustrates, in block-diagram form, an example embodiment,implemented by the wireless device 130, for preparing the media contentfor transmission and an example embodiment, implemented by the display120 a, for receiving the media content and preparing the received mediacontent for display on the display screen 212. The block-diagram of FIG.6A provides only a high-level illustration of components and steps used.Various other components and steps may be used. Additionally, the orderof the steps is only illustrative and is non-restrictive. The variouscomponents shown may be implemented as hardware-only components, forexample, using integrated circuit fabrication. The various componentsmay be implemented as software-only components, residing in memory andimplemented by a processor. Additionally, the various components may beimplemented using hardware and software components. For example, adedicated hardware media encoder, built using integrated circuitfabrication, may be controlled by software algorithms, for example,firmware. Accordingly, the various blocks are only illustrativefunctional blocks.

In FIG. 6A, the wireless device 130 transmits content 502 to the display120 a. The wireless device 130 accordingly acts a “source” of content502 for the display 120 a, whereas the display 120 a acts as a “sink”for the content 502 received from the wireless device 130. The wirelessdevice 130 may be referred to as a “source”, and the display 120 a maybe referred to as a “sink”. Other types of devices may function as a“sink”. For example, a speaker 120 c may receive audio content.Accordingly, the display 120 a may be replaced with the speaker 120 c(FIG. 1A) in some embodiments.

The content 502 may be stored in memory 320, for example as persistentdata 386, or on a removable memory card 332, or may be retrieved fromthe Internet 115 in real-time, or may be generated by the processor 302(such as graphical content generated by a video game, or by otherapplications). The content may include video, pictures, text and/orother information suitable for display on the display screen 212 of thedisplay 120 a and optionally audio suitable for playback using thespeaker 256 of the display 120 a, or audio suitable for playback usingthe speaker 120 c.

After session establishment at 410, the processor 302 of the wirelessdevice 130 may launch the P2P Streaming application 382. The P2PStreaming application 382 may control various aspects of transmission ofthe content 502, such as, controlling which content to transmit. In oneembodiment, the P2P streaming application 382 receives an indicationfrom a user, for example via an input device 306, that the content 502to be transmitted is any content shown on the display screen 312 of thewireless device 130. Accordingly, the P2P streaming application 382clones the display screen 312 onto the display screen 212 of the display120 a, which may have a larger screen. This allows for sharing thecontents of the display screen 312 with others (e.g., in the same room)using a larger screen. However, in some embodiments, the P2P Streamingapplication 382 may extract various aspects of the content displayed onthe display screen 212, such as sensitive information relating topasswords, or information marked as confidential (for example, bydetecting the word “confidential” being displayed on the display screen212).

The content 502 is sent to an encoder 504 for compression and encodingbased on the protocol and/or standard determined at the devicecapability negotiation phase 408. The compression scheme may be theH.264 video compression scheme as previously explained. The video-streammay be encoded into a plurality of Groups of Pictures (GOPs), where eachGOP has a set of frames. The GOP structure allows for compression of thevideo stream by using a number of different types of video frames, eachoffering different levels of compression. Each display frame is thuscomprised of multiple video frames. Each GOP includes only onekey-frame, the key-frame including a full representation of an imageassociated with the frame. Redundancy in the video-stream is removed byincluding predicted-frames in the GOP, which only include differenceinformation from reference-frames, such as the key-frame. Accordingly,for a given GOP, the more predicted-frames present, the greater thecompression. However, using too many predicted-frames may reduce thevideo-stream quality, as less scene information is included in thecompressed stream. For example, when a scene of video changes, i.e. theentire background is changed, a new key-frame may be included.

The video-stream may be implemented using Miracast, a peer-to-peerwireless screencast standard formed via Wi-Fi Direct connections.Miracast enables wireless or wired delivery of compressed standard orhigh-definition video to or from electronic devices. Typically, both thesending and receiving devices must be Miracast certified. However,Miracast adapters which plug into HDMI or USB ports are available whichallow streaming to a non-certified device. Miracast allows a portabledevice or computer to securely send up to 1080p HD video and 5.1surround sound (AAC and AC3 are optional codecs, mandated codec isLPCM—16 bits 48 kHz 2 channels).

Within the GOP, a first frame type of the different types, theIntra-Frame (“I-Frame” or “I” for short) includes full imageinformation, and may be used as a reference-frame. Accordingly, theI-Frame is the key-frame and is the largest frame, in terms of datasize. Thus, the I-Frame requires the longest transmission time. A secondframe-type is the Predicted-Frame (“P-Frame” or “P” for short), and isbased on the closest preceding I-Frame or P-Frame, and may be used as areference-frame. A P-Frame typically requires much less disk space thanan I-Frame of the same GOP, and thus requires less transmission timethan an I-Frame.

Examples of various GOP structures 420, 430 and 440 are represented inFIG. 5. GOP structures 420 and 430 GOP represent the same length ofvideo (in time) as each other. However, each structure has a differentnumber of frames of each type. Structure 420 represents one GOP havingone I-Frame and seven P-Frames. Structure 430 represents two GOPs eachhaving one I-Frame and three P-Frames. Since the I-Frames typicallyoffer less compression than the P-Frames, the GOP structure 430 mayoffer less compression than that of GOP structure 420. GOP structure 430requires two I-Frames and six P-Frames to represent the same length ofvideo as GOP structure 420, which instead requires one I-Frame and sevenP-Frames.

Structure 440 represents four GOPs having one I-Frame and one P-Frameeach. However, four GOPs are needed to represent the same length ofvideo as that of GOP structures 420 and 430. Accordingly, the GOPstructure 440 offers the least compression of all the representationsshown.

Each GOP may be characterized by a GOP size, i.e. the number of framesin the GOP, and by a GOP structure, i.e. the number of each type offrame in the GOP, and the arrangement thereof. Increasing the GOP sizeincreases the compression of the video-stream, reducing the time andbandwidth needed to transmit and process the video-stream; as thetime-period between two successive key-frames (i.e. I-Frames) isincreased, thus less key-frames are required. Adjusting the GOPstructure may affect the compression of the video-stream, as explainedpreviously; thus affecting the time and bandwidth needed to transmit andprocess the video-stream. Accordingly, the throughput of themedia-stream outputted by the encoder 504 may be directly correlatedwith both the GOP size and GOP structure, amongst other factors.

The encoder 504 may encapsulate one or more frames into packetsformatted according to an MPEG-transport stream (MPEG-TS) format. Anexample structure of an MPEG-TS formatted packet 610 is shown in FIG.6B. Each MPEG-TS packet includes header information 612 and a number ofFrames (I-, or P-Frames) 614. The header information 612 providesadditional data relating to the Frames 614, such as data forsynchronization and for maintaining transmission integrity when thetransmission signal is degraded. The MPEG-TS packet may be of a fixeddata size, such as 188 Bytes. Accordingly, the number of Framesencapsulated therein may depend on the type of frames encapsulated andtheir corresponding data sizes.

The encoder 504 may output, to the source buffer 506, a series ofMPEG-TS packets similar to the MPEG-TS formatted packet 610. The sourcebuffer 506 may be implemented as a First-In, First-Out (FIFO) structurestored in the RAM 308 of the wireless device 130. The source buffer 506may be allocated only a limited space in the RAM 308, sufficient to onlystore a pre-determined number of GOPs or Frames. The memory spaceallocated to the source buffer 506 may be based on limitationsassociated with the sink buffer 516 or other limitations.

The source buffer 506 outputs the frames stored therein to the WLANtransceiver 508 (which includes transmitter and receiver capability) forfurther processing and transmission. The WLAN transceiver 508 mayencapsulate several MPEG-TS packets into one Wi-Fi Direct (WFD) packet.An example structure of an WFD packet 620 packet 620 is shown in FIG.6B. The WFD packet 620 may include: (1) an IP header 622, for exampleproviding the IP address of the display 120 a; (2) a UDP header 624, forexample identifying the audio or video stream; (3) an RTP header 626providing time-stamp information to synchronize media playback, sequencenumber information to indicate the position in WFD packet in relation toother WFD packets, and other information; and (4) payload information,for example, a number of MPEG-TS packets 630-636, as previouslyexplained. The WFD packet 620 may have a fixed data size, such as 1356Bytes. The header information 622, 624, 626 may occupy only 40 Bytes.Accordingly, 1316 Bytes are allocated for pay-load information.Accordingly, one WFD packet 620 may include up to 7 MPEG-TS packets, asexplained in the example packet structures 610 and 620.

The WLAN transceiver 508 may transmit each WFD packet 620 as soon as itis ready for transmission, to be received at the WLAN transceiver 518 ofthe display device 120 a. The WLAN transceiver 518 may then extract theencapsulated MPEG-TS packets and send them to the sink buffer 516 toawait processing by the decoder 514. Frames 614 may be transmittedout-of-order, and thus may remain in the sink buffer 516 until theappropriate time for displaying the content 502 stored in the frame. Thedisplay 120 a then displays on the display screen 212 the decodedcontent as received.

Notice-of-Absence Scheduling

FIGS. 7A and 7B illustrate example Notice-of-Absence (NoA) schedules 700and 750, as set by a group-owner (GO) 702, in the time-domain. The NoAschedules 700, 750 define the time-allocated for communication betweenthe GO 702 and a client 704. The NoA schedule 700 includes eightperiods-of-absence 710-717 and seven periods-of-availability for mediatransmission 720-726 between the GO 702 and the client 704. The NoAschedule 750 includes five periods-of-absence 760-764 and fiveperiods-of-availability for media transmission 770-774 between the GO702 and the client 704. However, the illustrated NoA schedules 700, 750in FIGS. 7A and 7B are only a snap-shot of the communication between theGO 702 and a client 704 over a longer duration. Accordingly, the NoAschedules 700, 750 shown in FIGS. 7A and 7B may repeat for as long asession is established between the GO 702 and the client 704.

Additionally, as previously explained, either the GO 702 or the client704 can be the “source” of media content. The source may be, forexample, the wireless device 130. Similarly, either the GO 702 or theclient 704 can be the “sink” for the media content. The sink may be, forexample, the display 120 a.

The NoA schedule 700 allocates more time for media transmission than theNoA schedule 750, which has longer periods-of-absence defined in theschedule. The NoA schedule may be set by the GO based on constraintsassociated with either the GO 702, the client 704, or the wirelesstransmission medium. For example, the GO 702 may be a battery-operateddevice; accordingly to reduce power-consumption associated withwireless-transmission, the GO 702 may set a NoA schedule to allow forlonger periods-of-absence, as the GO 702 (and also the client 704) canenter a low-power state during the periods-of-absence. Accordingly, theNoA schedule 750 allows for more aggressive power savings than the NoAschedule 700. The GO 702 may also set the NoA-schedule to reduceinterference with other devices transmitting on the same frequency(i.e., wireless channel) as the GO 702 and the client 704. Accordingly,during the periods-of-absence, other devices may communicate with eachother, and during the periods-of-availability the other devices mayenter into periods-of-absence to allow the GO 702 and the client 704 tocommunicate.

Other limitations may include limitations associated with hardwareconstraints. The hardware of the GO 702 and/or the hardware of theclient 704 may include performance constraints; thus the media fortransmission must be encoded using parameters for encoding that requireless hardware performance. The hardware constraints may include any of asink buffer limitation for storing received data from a source, a sinkdecoder limitation for decoding received data from the source, a sourcebuffer limitation for storing data for sending to a sink, and a sourceencoder limitation for encoding data for sending to the sink.

Other limitations may include limitations associated with the media fortransmission. The media for transmission may only require a limitedbandwidth; thus, increasing the periods-of-absence may have no negativeeffect on the performance of the transmission. The performance of thetransmission may be determined by measuring the latency (i.e., delay dueto processing and transmission) in transmission of each frame. The NoAschedule can then be adjusted to allocate more and/or longerperiods-of-availability when the latency is above a threshold value, andadjusted to allocate more and/or longer period-of-absence when thelatency is below a threshold value.

Wireless Transmission of Real-Time Media

Reference is now made to FIGS. 8A and 8B, illustrating a flow-chart of amethod 800 for wireless transmission of a real-time media from a source,such as the wireless device 130, to a sink, such as the display 120 a,over a wireless transmission channel. The real-time media is encoded, aspreviously explained, and is made up of a number of frames. The method800 may be implemented by the wireless device 130 or other sourcedevice. The method 800 may be carried out by software executed, forexample, by a processor. Coding of software for carrying out such amethod 800 is within the scope of a person of ordinary skill in the artprovided the present disclosure. The method 800 may contain additionalor fewer processes than shown and/or described, and may be performed ina different order. Computer-readable code executable by the processor302 to perform the method 800 may be stored in a computer-readablemedium such as a memory of a host device.

In accordance with the method 800 shown in FIGS. 8A and 8B, the wirelessdevice 130 determines, based on a time-allocation for the wirelesstransmission, an available bandwidth for the wireless transmission and atime-required for transmission of a frame of a first frame-type. Thereal-time media is encoded for the wireless transmission such that thetime-required for transmission of the first frame-type is less than orequal to a period-of-availability for transmission defined in thetime-allocation for the wireless transmission. A frame of the firstframe-type is aligned, for transmission, with the start of theperiod-of-availability.

A peer-to-peer communication session is initiated at 802 between thewireless device 130 and the display 120 a. The session may be initiatedaccording to the steps 402-410 previously explained with reference toFIG. 4. Accordingly, the wireless device 130 and the display 120 a willdetermine a group-owner (GO) based on the GO Negotiation phase 406.Thus, the wireless device 130 implementing the method 800 can be eitherthe GO or the client.

When the wireless device 130 is the client, the wireless device 130receives the time-allocation from the GO, i.e. the display 120 a. Thetime-allocation may be defined by a Notice-of-Absence (NoA) schedule,such as the NoA schedules shown in FIGS. 7A and 7B. The NoA schedule maybe transmitted by the GO in a beacon frame, or in a Probe Responseframe, or in a NoA Action frame, and 25 received by the wireless device130.

When the wireless device 130 is the GO, the wireless device 130 sets thetime-allocation, for example, by defining a NoA schedule. The wirelessdevice 130 also sends the NoA schedule to the display 120 a and to anyother clients in a beacon frame, or in a Probe Response frame, or in aNoA Action frame. Thus, when the wireless device 130 is the GO, thewireless device 130 can set new NoA schedules dynamically. However, thewireless device 130 may have some limitations in setting thetime-allocation, for example, to avoid interference with devicesoperating on an interfering channel which are not under the control ofthe wireless device 130.

Any period-of-absence declared in the NoA schedule is a period duringwhich no wireless transmission is allowed by the GO. Accordingly, theavailable bandwidth for the wireless transmission is limited by theperiods-of-absence. At 804, the wireless device 130 then determines theavailable bandwidth (BW_(a)), based on a time-allocation for thewireless transmission. The wireless device 130 may transmit data packetsof a known data size to the display 120 a during aperiod-of-availability (PoA), and measure the time required to receivean acknowledgement of receipt of all the data packets at the display 120a from the time of sending the first data packet. The wireless device130 then determines the bandwidth during the period-of-availability. Thebandwidth during the period-of-availability is then adjusted by theprocessor 302 to compute an estimate of the available bandwidth takinginto consideration the duration and number of periods-of-absence, asreceived by the wireless device in the NoA schedule. The processor 302may additionally determine an estimated value for the availablebandwidth that also takes into consideration the periods-of-absence nowireless transmission is allowed in which case the estimate of theavailable bandwidth would be less than the estimate of the availablebandwidth taking into consideration only the period-of-availability.

The defined periods-of-availability in the NoA schedule can be used tosend payload data (i.e., media-frames) from the wireless device 130 tothe display 120 a. However, in some embodiments, as demonstrated in FIG.6B, the payload data is encapsulated in an MPEG-TS packet 610, which isfurther encapsulated in a WFD packet 620; therefore additional headerinformation, such as headers 612, 622, 624 and 626 also need to betransmitted. The header information associated with a particular mediaframe may be transmitted prior to transmission of the particular mediaframe. Accordingly, the available time-interval for transmission of themedia frames is shorter than the defined period-of-presence.

The encoder 504 determines the best encoding settings given theavailable bandwidth, the type of media for transmission, and thehardware capabilities of both the source and sink devices. Each set ofsettings for encoding the media requires different performancerequirements from each of the wireless device 130 and/or the display 120a. To accommodate different types of devices, the encoder settings maybe varied and are negotiated during the device capability negotiationphase 408 during the session initiation at 802. Thus, the wirelessdevice 130 determines a protocol and a set of settings for encoding themedia for wireless transmission suitable for both the wireless device130 and the display 120 a. The throughput constraint(s) may depend onthe negotiated scheme for encoding the media; as the display 120 a mayprovide information regarding hardware capabilities and constraints ofthe display 120 a to the wireless device 130 during the devicecapability negotiation phase 408. The throughput constraints areconsidered, and a maximum throughput constraint (Thpt_(HW)) isdetermined by the wireless device 130 at 806, based on one or morehardware limitation(s) associated with the wireless device 130 and/orthe display 120 a.

Hardware constraints may exists when, for example, the agreed uponencoding scheme requires high-definition video transmission. The sourcebuffer 506 and/or the sink buffer 516 may then limit the maximumthroughput at which the media can be transmitted and/or received.Similarly, the source encoder 504 and/or the sink decoder 514 may alsolimit the maximum throughput at which the media can be transmittedand/or received. Accordingly, the processor 302 may determine anestimated value for the maximum throughput based on the throughputconstraints for the wireless device 130 and the display 120 a.

Determining Ability to Control NoA Schedule

Based on the ability of the wireless device 130 to control thetime-allocation, the processing steps may vary. As previously explained,when the wireless device 130 is the client, the wireless device 130receives the time-allocation from the GO, i.e. the display 120 a.Accordingly, in some embodiments, the client device has no control overthe time-allocation. Additionally, in some embodiments, the GO may beconstrained in setting the NoA schedule due to additional limitations.The GO may be connected to one or more client devices imposingrestrictions on the GO, or the GO may be restricted in setting the NoAschedule due to the lack of availability of the wireless spectrum, forexample due to wireless interference.

At 820, the processor 302 determines if the wireless device 130 cancontrol the time-allocation. In one embodiment, it may be determinedthat the wireless device 130 can control the time-allocation if thewireless device 130 is the GO and has no limitations in setting the NoAschedule.

When the Wireless Device Cannot Control Time-Allocation (i.e., Client orConstrained GO)

When the wireless device 130 is a client device or a constrained GO, at820 the processor 302 will typically determine that the wireless device130 has no control over the time-allocation. The real-time media is thusencoded for the wireless transmission based on the available bandwidth(BW_(a)) or a throughput constraint (Thpt_(HW)). In such embodiments,the encoder 504 of the wireless device 130 will encode the real-timemedia for the wireless transmission based on the minimum of theavailable bandwidth (BW_(a)) and the throughput constraint (Thpt_(HW))at 826, to ensure that the encoded media does not require morethroughput than is available. When the available bandwidth (BW_(a)) isless than the maximum throughput constraint (Thpt_(HW)), the encoder 504of the wireless device 130 will encode the real-time media for thewireless transmission based on the available bandwidth (BW_(a)) at 826;since the available bandwidth (BW_(a)) is more limited than the maximumpossible throughput. However, when the available bandwidth (BW_(a)) isgreater than or equal to the maximum throughput constraint (Thpt_(HW)),the encoder 504 of the wireless device 130 will encode the real-timemedia for the wireless transmission based on the maximum throughputconstraint (Thpt_(HW)) at 826; since the maximum throughput constraint(Thpt_(HW)) is more limited than the maximum possible throughput. Thishelps to prevent any buffer overflow or underflow, whilst maximizing thequality of the real-time media on the display 120 a; as the leastpossible compression is used.

Additionally, at 826, the encoder 504 of the wireless device 130 encodesthe real-time for the wireless transmission to ensure than thetime-required for transmission (t_(TX)) of one key-frame; such as anI-Frame is less than or equal to the time of one period-of-availability(PoA) as defined in the NoA scheduled. A key-frame requires a longertransmission time than non-key frames, thus are prone to fragmentationduring transmission if the key-frame requires more time for transmissionthan is available in one PoA. Accordingly, the processor 302 maydetermine the maximum data-size that can be transmitted in one PoA. Theencoder 504 then adheres to this maximum data-size and ensures no framesexceed the determined maximum data-size.

For example, consider FIG. 9A, illustrating a NoA schedule 910, defininga NoA schedule having a PoA 901 starting at 912 and ending at 914.However, the PoA 901 is set by the GO and is shorter in time than thetime-required to transmit (t_(TX)) one I-Frame 902. Accordingly, thetransmission of the I-Frame 902 is fragmented and is only completedafter two PoA. This fragmentation may result in added latency intransmission, thereby causing the real-time media to lag on the display120 a. The encoder 504 will thus accommodate for the NoA schedule, asset by the GO. This is shown in FIG. 9B, showing the same NoA schedule910, having PoA 901 starting at 912 and ending at 914. However, theencoder 504 has now encoded an I-Frame 904 in accordance with the method800; thereby ensuring that the time-required for transmission of theI-Frame 904 is less than the time of one PoA as defined in the NoAschedule.

To encode the real-time media for wireless transmission, the encoder 504may modify various aspects of the encoding to ensure that the throughputrequirements do not exceed the throughput constraints and the bandwidthavailable. In one embodiment, the real-time media is made up ofconsecutive Groups of Pictures (GOPs), each GOP having a key-frame.Accordingly, the encoder 504 may adjust either the GOP structure and/orthe GOP size to achieve the required throughput. For example, withreference to FIG. 5, the key-frame is the I-Frame. As previouslyexplained, when switching the GOP structure 420 to the GOP structure430, the throughput required will be increased; as more I-Frames arepresent, which offer less compression than P-Frames. Similarly, whenswitching the GOP size 420 (i.e., eight frames per GOP) to the GOP size440 (i.e., two frames per GOP), the throughput required will beincreased; as more I-Frames are present for a given length of video (intime).

The real-time media is thus encoded in correspondence with the availablebandwidth and the throughput constraints. However, at any time, thetime-allocation can be changed by the GO (i.e., display 120 a), thuspossibly changing the available bandwidth. For example, when a secondP2P client device disassociates from the GO, the GO may adjust the NoAschedule in response to the dissociation. The wireless device 130 willthus monitor for a new time-allocation for wireless transmission. When anew time-allocation is detected, the wireless device 130 will determinea new available bandwidth for the wireless transmission based on the newtime-allocation, by repeating the method 800 starting at 804. The mediais then encoded for the wireless transmission based on the new availablebandwidth.

During the wireless transmission, at 828, the WLAN transceiver 508aligns the transmission of the key-frame, such as the I-Frame, with thestart of the period of availability. For example, with reference to FIG.9B, the transmission of the I-Frame 904 begins at the start of the PoA901 at 912. By aligning the transmission of the I-Frame 904 with thestart of the PoA 901, the chance of an incomplete transmission of theI-Frame 904 is reduced. The reduced number of fragmented frames willthus help improve the latency in transmission of the real-time media.

When the Wireless Device Controls Time-Allocation (i.e., Client orConstrained GO)

When the wireless device 130 is the GO, the wireless device 130 sets thetime-allocation using a NoA schedule. Additionally, in some embodiments,when the wireless device 130 is a client device, the wireless device 130is still able to control the time-allocation by sending a request to theGO (i.e., the display 120 a) to modify the NoA schedule, for example, toallow for more periods-of-presence. The request may be sent using a P2PPresence Request, and the GO may then determine whether to accept therequest when received. When accepted, the GO will adapt a modified NoAschedule, as requested by the client. Accordingly at 820, the processor302 may determine that the wireless device 130 is able to control thetime-allocation both when the wireless device 130 is the GO or theclient.

In such embodiments, the encoder 504 of the wireless device 130 willencode the real-time media for the wireless transmission based on theminimum of the available bandwidth (BW_(a)), the hardware throughputconstraint (Thpt_(HW)), or any other throughput constraints at 826, toensure that the encoded media does not require more throughput than isavailable. However, since available bandwidth (BW_(a)) is the bandwidthas adjusted for period-of-absence defined in the NoA schedule and sincethe NoA schedule can be adjusted by the wireless device 130, theavailable bandwidth (BW_(a)) used in 832 may be considered to be thebandwidth of the wireless transmission channel with noperiods-of-absence defined. The encoder is thus able to utilize thebest-available encoding, given the throughput limitations of thewireless device 130 and the wireless channel; as the wireless device 130is able to control the NoA schedule to provide sufficient bandwidth forwireless transmission.

At the optional step 833, the processor 302 of the wireless device 130determines the flexibility in setting a dynamic NoA schedule. The NoAschedule may be easily modifiable, for example, when only one client isconnected to the wireless device; i.e. the display 120 a. The wirelessdevice 130 may dynamically modify the NoA schedule based on theframe-type of the frame that is queued for transmission, as will beexplained. However, in other embodiments, the GO may not be able tomodify the NoA frequently, for example, when multiple client devices areconnected thereto. Additionally, it may not be desirable to modify theNoA schedule frequently, as additional processing and signaling isrequired.

No Dynamic NoA Scheduling

When no dynamic NoA scheduling is implemented, the wireless device 130sets one NoA schedule for all frame types. The NoA schedule may howeverbe modified to better accommodate new clients, newly detectedenvironmental conditions, or any other constraint.

The processor 130 of the wireless device, at 846, determines thetime-required for transmission (t_(TX)) of one I-Frame, as encoded atstep 832. In some embodiments, the time-required can be computed basedon the bandwidth of the wireless channel and the average size of anI-Frame. Due to the encoding by the encoder 504, all frames of the sameframe-type (i.e. I-Frames) having the same compression characteristicsand settings are expected to have a substantially similar data-size toeach other; thus requiring a substantially similar transmission time toeach other. In other embodiments, the time-required can be computedbased on the bandwidth of the wireless channel and the maximum size ofan I-Frame.

As previously explained, frames of each frame-type are expected to havea different average data-size. The I-Frame is a key-frame, whichprovides a reference to P-Frames; thus the I-Frame is expected to occupythe larger data-size. Other types of frames may be used, the firstframe-type having a larger data-size and requiring a longer transmissiontime than the second frame-type.

The wireless device 130 then sets a new time-allocation at 848 bydefining a new NoA schedule. The new NoA schedule definesperiods-of-availability (PoA) such that each period of availability isequal to in duration or longer than the time-required for transmission(t_(TX)) of one I-Frame. This ensures that the I-Frames; i.e. thekey-frame having the largest data-size and requiring the longest periodfor transmission is not fragmented during transmission. In someembodiments, the PoAs are defined such that the duration of each PoA isan integer multiple of the time-required for transmission (t_(TX)) ofone I-Frame. Additionally, the NoA schedule is set such that the PoAsdefined allow for a sufficient throughput for transmission of thereal-time media in real-time or near real-time. In one example, thewireless channel offers 100 Mbps of throughput and the media is encodedat a bandwidth of 50 Mbps. Accordingly, the NoA schedule is defined suchthat the PoAs occupy at least 50% of the time, or slightly more time, toallow for dropped packets to be retransmitted and for and overhead.

The wireless device 130 then aligns the I-Frame for transmission withthe start of a PoA at 850. The transmission of the I-Frame beginsimmediately or soon after the start of a PoA, to enhance the chance ofcomplete transmission of the I-Frame within one PoA is successful,requiring no fragmentation of the I-Frame.

Reference is made to FIGS. 9A and 9C, illustrating NoA schedules 910 and930, respectively. In each case, the same I-Frame 902 is transmitted. InFIG. 9A, the NoA schedule 910 defines PoA 901 that is too short forcomplete transmission of the I-Frame 902, as previously explained.However, the NoA schedule 930 defines PoA 903 that is longer in durationthan the time-required for transmission of the I-Frame 902. In FIG. 9C,the I-Frame 902 begins transmission at the start of the PoA 903, at 932,and ends transmission prior to the end of the PoA 903, at 934.Additionally, three P-Frames are also transmitted after the I-Frame 902in the same PoA 903. The increased duration of the PoA 903 in comparisonwith the PoA 901 has prevented fragmentation of the I-Frame, thusreducing latency in transmission. Additionally, the transmission of theI-Frame 902 is aligned with the start of the PoA 903, at 932, i.e. theI-Frame is transmitted immediately at the start of the PoA 903, at 932.This reduces the chance that the I-Frame may be fragmented.

Dynamic NoA Schedule

When dynamic NoA scheduling is implemented, the wireless device 130 setsa NoA schedule depending on the frame-type of the frame queued fortransmission. In one embodiment, one in eight frames is an I-Frame, forexample, as defined in GOP structure 420 in FIG. 5. Accordingly, a firstNoA schedule may be set for the transmission of the I-Frame, then asecond NoA schedule set for the transmission of the seven P-Frames, thenthe first NoA schedule reverted to.

At 834, the processor 302 of the wireless device 130 determines theframe-type of the frame queued for transmission in the buffer 506, forexample by examining the contents of the frame and/or the size of theframe. This information is used to then determine the time-required fortransmission (t_(TX)) of the frame of that particular type, similarly tostep 846. For example, if an I-Frame is queued, the time-required fortransmission (t_(TX)) of an I-Frame is determined, however, if a P-Frameis queued, the time-required for transmission (t_(TX)) of a P-Frame isdetermined. A new NoA is then set at 838, definingperiods-of-availability (PoA) such that each period of availability isequal to in duration or longer than the time-required for transmission(t_(TX)) of one frame of the determined frame-type. Additionally, theNoA schedule is set such that the PoAs defined allow for a sufficientthroughput for transmission of the real-time media in real-time or nearreal-time. The wireless device 130 then aligns the frame fortransmission with the start of a PoA at 840. Steps 834, 836, 838 and 840are then repeated for each frame, however; a new NoA schedule is setonly when the frame-type changes. Accordingly, if only I- and P-Framesare used, two NoA schedules may be set in alternating order.

An example of a dynamic NoA schedule 940 is shown in FIG. 9D. A long PoA905 is defined to start at 942 and end at 944. During the long PoA 905,I-Frame 902 is transmitted at the start of the PoA 905, followed bythree P-Frames. The long PoA 905 is followed by a period-of-absence,then by a short PoA 907, which starts at 946 and ends at 948. In theshort PoA 907, four P-Frames are transmitted. Accordingly, in one longPoA 905 and one short PoA 907 an entire GOP having GOP structure 420 istransmitted. By utilizing the short PoA 907, the wireless device 130 isable to enter a low power state for longer periods, thus reducing thepower consumption.

General

The above-described embodiments of the present disclosure are intendedto be examples only. Those of skill in the art may affect alterations,modifications and variations to the particular embodiments withoutdeparting from the scope of the application. Although the descriptionrelates to specific examples for illustration, where the WLAN is an IEEE802.11-based network, for example, different environments may beapplicable as well. As a few other examples, the wireless networking maybe based on a WiMAX network (i.e. IEEE 802.16), or an Ultra-WideBand(UWB) network (i.e. IEEE 802.15). The invention described herein in therecited claims intends to cover and embrace all suitable changes intechnology.

The steps and/or operations in the flowcharts and drawings describedherein are for purposes of example only. There may be many variations tothese steps and/or operations without departing from the teachings ofthe present disclosure. For instance, the steps may be performed in adiffering order, or steps may be added, deleted, or modified.

While the present disclosure is described, at least in part, in terms ofmethods, a person of ordinary skill in the art will understand that thepresent disclosure is also directed to the various components forperforming at least some of the aspects and features of the describedmethods, be it by way of hardware components, software or anycombination of the two, or in any other manner. Moreover, the presentdisclosure is also directed to a pre-recorded storage device or othersimilar computer readable medium including program instructions storedthereon for performing the methods described herein.

The present disclosure may be embodied in other specific forms withoutdeparting from the subject matter of the claims. The described exampleembodiments are to be considered in all respects as being onlyillustrative and not restrictive. The present disclosure intends tocover and embrace all suitable changes in technology. The scope of thepresent disclosure is, therefore, described by the appended claimsrather than by the foregoing description. The scope of the claims shouldnot be limited by the preferred embodiments set forth in the examples,but should be given the broadest interpretation consistent with thedescription as a whole.

The invention claimed is:
 1. A method for wireless transmission of areal-time media from a source device to a sink device over a wirelesstransmission channel, wherein the real-time media comprises a pluralityof frames, the method comprising: determining an available bandwidthbetween the source device and the sink device for the wirelesstransmission based on a time-allocation of the source device forwireless transmission defined by a Notice-of-Absence schedule;determining a time-required for wireless transmission of a frame of afirst frame-type based on the time-allocation of the source device forwireless transmission defined by the Notice-of-Absence schedule; andencoding the real-time media into the first frame-type for wirelesstransmission based on a minimum of the available bandwidth between thesource device and the sink device or a throughput constraint between thesource device and the sink device and such that the time-required fortransmission of the frame of the first frame-type is less than or equalto the time-allocation for the wireless transmission.
 2. The method ofclaim 1, further comprising: aligning a start of transmission of theframe of the first frame-type with a start of the time-allocation. 3.The method of claim 1, further comprising: determining, based on ahardware limitation associated with one or both of the source device andthe sink device, the throughput constraint for the wirelesstransmission; when the available bandwidth is less than the throughoutconstraint, encoding the real-time media for the wireless transmissionbased on the available bandwidth; and when the available bandwidth isgreater than or equal to the throughput constraint, encoding thereal-time media for the wireless transmission based on the throughputconstraint.
 4. The method of claim 1, further comprising setting thetime-allocation of the source device for wireless transmission using theNotice-of-Absence schedule.
 5. The method of claim 1, further comprisinginitiating a peer-to-peer (P2P) communication session between the sinkdevice and the source device.
 6. The method of claim 1, wherein thereal-time media is a Miracast video-stream.
 7. The method of claim 1,wherein the time-allocation is a period-of-availability defined by theNotice-of-Absence schedule.
 8. The method of claim 1, wherein thethroughput constraint is a maximum throughput constraint between thesource device and the sink device.
 9. A method for wireless transmissionof a real-time media from a source device to a sink device over awireless transmission channel, wherein the real-time media comprises aplurality of frames, the method comprising: determining an availablebandwidth between the source device and the sink device for the wirelesstransmission based on a time-allocation of the source device forwireless transmission defined by a Notice-of-Absence schedule; when aframe of a first frame-type is queued for transmission, determining atime-required for wireless transmission of the frame of the firstframe-type queued for transmission based on the available bandwidth;encoding the real-time media into the first frame-type for wirelesstransmission based on a minimum of the available bandwidth between thesource device and the sink device or a throughput constraint between thesource device and the sink device; and setting a new time-allocation forwireless transmission such that the time-allocation of the source devicefor wireless transmission is greater than or equal to the time-requiredfor transmission of the first frame-type.
 10. The method of claim 9,further comprising: determining a frame-type of a frame queued fortransmission, the frame-type being either the first frame-type or asecond frame-type; when a frame of the second frame-type is queued fortransmission, determining a time-required for wireless transmission ofthe frame of the second frame-type queued for transmission based on theavailable bandwidth; encoding the real-time media into the secondframe-type for the wireless transmission based on a minimum of theavailable bandwidth between the source device and the sink device or athroughput constraint between the source device and the sink device; andsetting a new time-allocation for wireless transmission such that thetime-allocation of the source device for wireless transmission isgreater than or equal to a time-required for transmission of the secondframe-type.
 11. The method of claim 10, wherein the first frame-type isan Intra-Frame, and the second frame-type is a Predicted-Frame.
 12. Themethod of claim 9, wherein the time-allocation is aperiod-of-availability defined by the Notice-of-Absence schedule. 13.The method of claim 9, wherein the throughput constraint is a maximumthroughput constraint between the source device and the sink device. 14.The method of claim 9, further comprising: determining, based on ahardware limitation associated with one or both of the source device andthe sink device, the throughput constraint for the wirelesstransmission; when the available bandwidth is less than the throughputconstraint, encoding the real-time media for the wireless transmissionbased on the available bandwidth; and when the available bandwidth isgreater than or equal to the throughput constraint, encoding thereal-time media for the wireless transmission based on the throughputconstraint.
 15. The method of claim 9, further comprising: aligning astart of transmission of the frame of the first frame-type with a startof the time-allocation.
 16. A wireless communication device, comprising:a processor; a memory storing media and instructions for wirelesstransmission of the media from a source device to a sink device over awireless transmission channel, wherein the processor is configured to:determine an available bandwidth between the source device and the sinkdevice for the wireless transmission based on a time-allocation of thesource device for wireless transmission defined by a Notice-of-Absenceschedule; determine a time-required for wireless transmission of a frameof a first frame-type based on the time-allocation of the source devicefor wireless transmission defined by the Notice-of-Absence schedule; andencode the real-time media into the first frame-type for wirelesstransmission based on a minimum of the available bandwidth between thesource device and the sink device or a throughput constraint between thesource device and the sink device and such that the time-required fortransmission of the frame of the first frame-type is less than or equalto the time-allocation for the wireless transmission.
 17. The wirelesscommunication device of claim 16, wherein the processor is furtherconfigured to align a start of transmission of the frame of the firstframe-type with a start of the time-allocation.
 18. The wirelesscommunication device of claim 16, wherein the processor is furtherconfigured to: determine, based on a hardware limitation associated withone or both of the source device and the sink device, the throughputconstraint for the wireless transmission; when the available bandwidthis less than the throughput constraint, encode the real-time media forthe wireless transmission based on the available bandwidth; and when theavailable bandwidth is greater than or equal to the throughputconstraint, encode the real-time media for the wireless transmissionbased on the throughput constraint.
 19. The wireless communicationdevice of claim 16, wherein the processor is further configured to setthe time-allocation of the source device for wireless transmission usingthe Notice-of-Absence schedule.
 20. The wireless communication device ofclaim 16, wherein the processor is further configured to initiate apeer-to-peer (P2P) communication session between the sink device and thesource device.
 21. The wireless communication device of claim 16,wherein the real-time media is a Miracast video-stream.
 22. A wirelesscommunication device, comprising: a processor; a memory storing mediaand instructions for wireless transmission of the media from a sourcedevice to a sink device over a wireless transmission channel, whereinthe processor is configured to: determine an available bandwidth betweenthe source device and the sink device for the wireless transmissionbased on a time-allocation of the source device for wirelesstransmission defined by a Notice-of-Absence schedule; when a frame of afirst frame-type is queued for transmission, determine a time-requiredfor wireless transmission of the frame of the first frame-type queuedfor transmission based on the available bandwidth; encode the real-timemedia into the first frame-type for wireless transmission based on aminimum of the available bandwidth between the source device and thesink device or a throughput constraint between the source device and thesink device; set a new time-allocation for wireless transmission suchthat the time-allocation of the source device for wireless transmissionis greater than or equal to the time-required for transmission of thefirst frame-type.
 23. The wireless communication device claim 22,wherein the processor is further configured to: determine a frame-typeof a frame queued for transmission, the frame-type being either thefirst frame-type or a second frame-type; when a frame of the secondframe-type is queued for transmission, determine a time-required forwireless transmission of the frame of the second frame-type queued fortransmission based on the available bandwidth; encode the real-timemedia into the second frame-type for the wireless transmission based ona minimum of the available bandwidth between the source device and thesink device or a throughput constraint between the source device and thesink device; and set a new time-allocation for wireless transmissionsuch that the time-allocation of the source device for wirelesstransmission is greater than or equal to a time-required fortransmission of the second frame-type.
 24. The wireless communicationdevice of claim 23, wherein the first frame-type is an Intra-Frame, andthe second frame-type is a Predicted-Frame.
 25. A non-transitory machinereadable medium having tangibly stored thereon executable instructionsfor execution by a processor of a wireless communication device toperform a method for encoding real-time media for wireless transmissionfrom a source device to a sink device over a wireless transmissionchannel, wherein the executable instructions, when executed by theprocessor of the wireless communication device, cause the processor to:determine an available bandwidth between the source device and the sinkdevice for the wireless transmission based on a time-allocation of thesource device for wireless transmission defined by a Notice-of-Absenceschedule; determine a time-required for wireless transmission of a frameof a first frame-type based on the time-allocation of the source devicefor wireless transmission defined by the Notice-of-Absence schedule; andencode the real-time media into the first frame-type for wirelesstransmission based on a minimum of the available bandwidth between thesource device and the sink device or a throughput constraint between thesource device and the sink device and such that the time-required fortransmission of the frame of the first frame-type is less than or equalto the time-allocation for the wireless transmission.
 26. Anon-transitory machine readable medium having tangibly stored thereonexecutable instructions for execution by a processor of a wirelesscommunication device to perform a method for encoding real-time mediafor wireless transmission from a source device to a sink device over awireless transmission channel, wherein the executable instructions, whenexecuted by the processor of the wireless communication device, causethe processor to: determine an available bandwidth between the sourcedevice and the sink device for the wireless transmission based on atime-allocation of the source device for wireless transmission definedby a Notice-of-Absence schedule; when a frame of a first frame-type isqueued for transmission, determine a time-required for wirelesstransmission of the frame of the first frame-type queued fortransmission based on the available bandwidth; encode the real-timemedia into the first frame-type for wireless transmission based on aminimum of the available bandwidth between the source device and thesink device or a throughput constraint between the source device and thesink device; set a new time-allocation for wireless transmission suchthat the time-allocation of the source device for wireless transmissionis greater than or equal to the time-required for transmission of thefirst frame-type.